5 inputs:
7 time series outputs (each with 512 timepoints), and maximum/minimum pressure.
Observations are noise-free.
For 1st wave, used an initial design of 200 points. Had emulators trained on this prior to the 3 hours, and just calculated implausibility once received actual observations.
Plotting ensemble and obs for the 7 flows:
Glancing at some LOOs:
We don’t have any error, but need to set it at something. For now, just set as diagonal.
What does NROY look like?
Plotting the known runs that are in NROY:
Sampled a wave 2 restricted to runs we had available already. Plotting:
Refitting basis, emulators to new simulations. Reduced tolerance to error to force a smaller NROY space (size is arbitrary, only interested in `best’ runs):
Plotting W1+W2 distributions + overlaying truth:
Part of the perceived improvement at W2 could just be because reduced our error tolerance.
The 10 runs with lowest implausibility at wave 2:
## f2 f3 fs2 fs3 alpha NROY_w2 NROY
## 1273 -0.09543336 -0.3480362 0.1631990 0.441243493 -0.3836307 TRUE TRUE
## 16182 -0.06373543 -0.3346185 -0.4355780 0.426106814 -0.3131207 TRUE TRUE
## 37699 -0.88673586 -0.3529626 0.7737754 -0.001525908 -0.2797586 TRUE TRUE
## 55417 -0.45261725 -0.3378206 0.8103676 0.191431676 -0.3764120 TRUE TRUE
## 61084 -0.01170803 -0.3497713 0.2576587 0.048704205 -0.2934673 TRUE TRUE
## 61158 -0.37902714 -0.3614865 0.9307290 0.204612706 -0.3064093 TRUE TRUE
## 68862 -0.58123997 -0.3593306 0.6799814 -0.016873887 -0.2807075 TRUE TRUE
## 73043 -0.08748638 -0.3269073 -0.7270260 0.583254196 -0.3643647 TRUE TRUE
## 76468 0.26808217 -0.3689451 -0.5316870 0.287363859 -0.3376002 TRUE TRUE
## 99134 -0.92063467 -0.3593677 0.9693147 0.171426453 -0.2742417 TRUE TRUE
Due to time constrains and lack of reliable access to Matlab or the internet (!!!), rather than sampling a new wave 3 and running ~100 new simulations, refitting emulators, and using these to get a single ‘best’ \(\textbf{x}^*\), instead stopped emulation here.
Took the 10 runs that minimised W2 implausibility, and ran these on the simulator.
Do these improve vs best run we’ve seen so far? How close can we get to the observations?
Plot the 10 new simulations:
Plotting these 10 runs on top of best run observed at waves 1 (blue) and 2 (purple):
Looking at true implausibility for 10 new runs, 4 are better than the best run we saw in W2:
## [1] 1774.4139 340.2810 1006.4356 1949.9291 303.2708 404.7306 935.7751
## [8] 565.2265 105.9921 1433.5852
Still had a bit of time, didn’t want to fit new emulators, so just perturbed the parameters around the current best run, and did 10 more simulations:
Looking at error for 10 new runs, 1 improves on best at W3:
## [1] 397.32957 708.25180 1038.29911 280.64751 48.13425 392.97887
## [7] 814.33111 481.11526 838.65244 330.32110
Adding the new best run to the earlier residual plot:
Just plotting W2/3/4 to see improvements:
What if we had emulated max/min pressure as well?
Look at residual vs true pressure are for the ‘best’ flow runs identified earlier (shown for W1-4):
## max_pressure min_pressure
## 1.2940584 0.3943566
## max_pressure min_pressure
## 0.1580628 0.5570256
## V3585 V3586
## 0.1830634 -0.2822653
## V3585 V3586
## -0.0669366 0.2347347
So in fact simply considering flows, we still improved our (average) fit to the max/min pressure each time.
Have we already found 1 of the closest runs from the 100k? 10 closest points in input space to the truth:
## [1] 76468 62884 1098 37337 49477 29393 82240 11665 86491 45249
And in fact, sample 76468 was actually selected as one of the best 10 after wave 2, hence was simulated at wave 3; i.e. given we restricted to choosing from 100,000 inputs, we successfully identified the best possible choice (by emulating flow only, hence adding pressure cannot have improved these results).